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Introduction 


PISA (Programme for International Student Assessment) is an interna- 
tional examination consisting of multiple-choice questions and open-ended 
questions periodically organized by the Organization for Economic Coop- 
eration and Development to measure the success of secondary education 
students of member countries in interpreting and applying what they learn 
at school to real life. Lots of middle school students in different countries 
don't get high scores on the PISA because PISA is an examination that tries 
to find out if students are well prepared for future challenges by analyzing, 
reasoning and communicating effectively, skills that demonstrate the capacity 
to continue learning in their future lives (Celen, Celik, Seferoglu, 2011). The 
students level of problem solving skills and their skills in integrating scientific 
principles and theories to real life situations, rather than just memorizing, is 
the main purpose of the PISA. The education systems of many countries, based 
on just the memorization of facts and principles, do not prepare students to 
do well on the PISA examination. 

Although primary education and secondary education students are ex- 
posed to excessively exam- and test-oriented teaching at schools and private 
teaching institutions in Turkey, the country took almost last place in sciences 
in the 2009 PISA among 65 countries (PISA, 2009). That indicates that there is 
something deficient or wrong in educational practices conducted in Turkey. 
What is wrong with educational practices conducted at schools in Turkey? 
What is the solution? According to Keeley (2008), even the most interesting 
activities or lessons may result in little or no conceptual understanding, when 
the ideas, prior knowledge, and readiness levels of students are ignored. 
In such cases, teaching may yield no conceptual understanding in science 
lessons. As a result, some gaps emerge between learning and teaching. 
These gaps come to light in national (e.g. level determination examinations 
[placement examination] and transition to higher education examinations) 
and international (e.g., PISA, TIMMS) examinations aimed at determining the 
success levels of students. However, when these examinations are taken, it is 
too late to return and make adaptations in teaching (Keeley, 2008). 
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Abstract. This study had three main 
purposes, to (a) determine students’ 
understanding of basic physics concepts, 
(6) analyze the science concepts and their 
connection to real-life context in science 
curricula and teaching materials, (c) evalu- 
ate effect of formative assessment probes 
on students’ understanding of selected 

key concepts. Subjects were 197, 8th grade 
students who had studied five basic physics 
concepts. To evaluate students’ understand- 
ing, a questionnaire consisting of formative 
assessment probes was administered with 
4 multiple choice and 4 open-ended ques- 
tions. Findings indicated that students had 
poor conceptual understanding of the con- 
cepts. Science curriculum, analyzed in terms 
of application to real-world contexts, indi- 
cated that textbooks included few practical 
explanations. In an intervention, one public 
school received formative assessment 
probes integrated with extra-curricular 
hands-on science instruction. Pre-post data 
showed significant increases in students’ 
understanding of basic physics concepts. 
Finding that these students significantly 
out-performed other groups implies: (a) 

the need to include students’ prior learning 
in instruction and (b) the need for more 
learning time to apply understanding of 
concepts to real-world contexts. 
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According to studies in related literature, the fundamental problem is that assessment practices, which 
are not integrated into teaching, do not promote students’ conceptual understanding (Black, 1993; McConnell, 
Steer, Owens, 2003; Yin, Shavelson, Ayala, Ruiz-Primo, Brandon, Furtak, 2008; Cliff, Freeman, Hansen, Kibble, Peat, 
& Wenderoth, 2008; Yin, Tomita, & Shavelson, 2013) End-of-unit or end-of-semester tests involving traditional 
assessment and evaluation methods, such as true - false questions, matching practices, filling in the blanks, and 
multiple-choice questions, encourage surface learning and rote learning as they usually cover isolated or de- 
tailed information likely to be forgotten by children in a short time (Butler, 1987; Butler & Neuman, 1995). Since 
these kinds of assessment techniques such as; true-false questions, matching practices, filling in the blanks, and 
multiple-choice questions which are capable of measuring the acquisitions gained by root learning and low- 
level acquisitions in general usually focus on grading, learning function stays in the background (Black, 1993; 
Black & William, 1998; Crooks, 1988). According to Duschl and Gitomer (1997), students take the easy way out 
by thinking, “it is enough that | pass the course” through evading the teaching activities that wrack their brains 
and require power of thinking in any education system, where knowledge and skills such as reasoning and 
critical thinking are not much regarded and assessed. Apart from that, the above-mentioned traditional assess- 
ment and measurement approaches have a negative effect on in-class learning climate by inciting competition 
instead of cooperation among students (Crooks, 1988; Yin, et al., 2008). This is because; the success of a student 
in the lesson is determined by the grades s/he gets and his/her rank among his/her friends. This being the case, 
students evaluate their performances through comparison with their friends instead of focusing on and evalu- 
ating their individual developments. The negative effects of such a classroom climate on the successes and the 
motivation of students are reported to be as follows: a) it is believed that students with low achievement levels 
are incompetent (Siero & Van Oudenhoven, 1995); b) it is accepted that ability and intelligence are innate and 
unchangeable, which discourages students from learning and improving themselves in the future (Vispoel & 
Austin, 1995); c) students avoid seeking help from their teachers or friends for fear that the questions they ask 
may be regarded as evidence of their inability and failure (Blumenfeld, 1992; Crooks, 1988). All these factors 
cause students to lose confidence in their own learning capacities by diminishing their motivation. 

When assessment is mentioned, written and oral examinations, as well as the grades achieved in them, 
come to mind in general. Thus, assessment is mostly used synonymously with written and oral examinations and 
assignments. According to Atkin, Coffey, Moorthy, Sato, and Thibeault (2005) the use of assessment synonymously 
with the above-mentioned examination types degrades the complex nature, stages, and purpose of assessment. 
This is because: grading is the smallest piece of assessment. However, assessment is quite a comprehensive word. 
It is at the top of Bloom’s taxonomy (Forehand, 2005) and requires advanced performance. What is important in 
assessment is to make quantitative and qualitative analyses aimed at determining the points students under- 
stand, the points they have deficiencies in, and the sorts of misconceptions they have. 

Any assessment that does not pursue the goal of grading, but is made for learning and teaching is a for- 
mative assessment (Keeley, 2011, 2012). What is meant by “assessment made for learning” is to determine what 
students know about the topic taught. The assessment for teaching, on the other hand, refers to adaptation of 
the coverage of the topic in the light of the information collected from students (Black, 1993; Yin, et al., 2008; 
Furtak, 2012; Yin et al., 2013). In formative assessment, the prior knowledge and the misconceptions of students 
guide the manner of covering the lesson. Within the scope of this approach, assessment is made before and 
during teaching in order to eliminate students’ imperfect and inadequate knowledge (Bell & Cowie, 2001; Black 
& William, 2009). Since formative assessment is also conducted during teaching, it provides both students and 
teachers with feedback concerning the learning and the teaching of the lesson. If the coverage of the lesson is 
adapted in accordance with such feedback obtained from students, the new coverage of the lesson may result in 
improved conceptual learning by students (Black, Harrison, Lee, Marshall, & William, 2004; Ruiz-Primo & Furtak, 
2004; Black & William, 2009). Hogan, Nastasi, and Pressley (2000) found that questioning helped students make 
explanations more complete and phrased in scientific terms. The primary objective of formative assessment is to 
improve student conceptual understanding during lessons. Thus, an assessment may be considered a formative 
assessment only if the information collected from students is used in the conduct of lessons (Keeley, 201 1:2012). 
In this regard, it is not a formative assessment unless the information collected from a level determination ex- 
amination regarding what students know, what students do not know, and what students know imperfectly is 
used in the teaching of a lesson. It is an integrated circular process that applies student learning data relatively 
immediately to enhance learning effectiveness. 

Many efforts have been made for formative assessment to be carried into effect in science teaching as of 
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the late 1990s. It has also been highlighted in documents determining national education policies (National 
Research Council, 2001, 2007, 2011) in the United States of America. Educational directorates and schools in 
many states of the USA treat formative assessment as a teaching approach (Cizek, 2010). In addition, formative 
assessment is integrated into many curricula (Lawrence Hall of Science, 2000), and publishing houses develop 
probes and tests based on formative teaching approaches for schools (Keeley, Eberle, & Farrin, 2005; Keeley 
& Harrington, 2010; Shepard, 2008). In his book titled “Educational Psychology” (1968), David Asubel said, “If / 
had to reduce all of educational psychology to just one principle, | would say this: The most important single fac- 
tor influencing learning is what the learner already knows. Ascertain this and teach him accordingly (p.6.)” The 
comprehensive literature review conducted by Black and William in 1998 concluded that formative assessment 
had a very positive effect on the learning and the motivation of students. According to Furtak (2012), formative 
assessment supports the learning of students, and its importance is gradually increasing in reformation of sci- 
ence lessons taught in classrooms. 

Previous studies show that students have different ideas in regard to science concepts. The ideas, which 
are based on simple observations in the course of time, but are not obtained through research and following a 
particular method based on real reasons, are called casual knowledge or misconception (Schnotz, Vosniadou, 
& Carretero, 1999). Such prior knowledge possessed by students that mostly conflicts with scientific knowledge 
is referred to as “misconception” (NRC, 2001). Misconceptions may result from limited observations and experi- 
ences of students in daily life or imperfect knowledge provided by books or hearsay and unsubstantiated tradi- 
tion (Asci, Ozkan, & Tekkaya, 2001; Capa, 2000; Sungur, Tekkaya, & Geban, 2001). According to Keeley (2012), 
misconceptions are misunderstood or misused by many educators. Perceiving the concepts held by children 
as misconceptions and treating them as an obstacle before education (Kizilcik & Gunes, 2011) are regarded as 
the misconceptions of educators concerning teaching. Today, the prior knowledge of students is regarded as a 
starting point for conceptual understanding. For instance, Larkin (2012) found that while pre-service teachers 
considered the misconceptions of students an obstacle before science teaching and a mistake to be corrected 
at the beginning of the semester, they noticed at the end of the semester that the misconceptions of students 
were important keys for opening the potential for teaching. In consideration of the fact that new learning cannot 
be achieved through direct transfer in science course, when roots of concepts are very deeply held by students, 
it is quite difficult to ensure conceptual understanding by students, unless correct knowledge is brought to light 
and discussed explicitly in the classroom (Posner, Stike, Hewson, & Gertzog, 1982). 

The new Turkish National Science Curriculum is designed in a cyclical way, in that each year students study 
the same concept but at a deeper level to develop more complex understanding through eliminating miscon- 
ceptions (MEB, 2005). However, recent research studies in Turkey indicate that middle and high school students 
(Koray & Tatar, 2003; Yumusak, Turkoglu, Aycan, & Aycan, 2004; Kocakulah & Acil, 2011; Ayvaci, Bakici, & Yildiz, 
2012) and also preservice teachers (GOnen, 2008; Yumusak, 2008; Tung, Akcam, & Dokme, 2012) all have diffi- 
culty understanding basic physics concepts. For example, Ayvaci et al., (2012) conducted a study on in-service 
middle school teachers, in-service high school teachers, and pre-service science teachers. They determined that 
research participants had low levels of understanding of concepts about mass, weight, and gravity, and they 
had many misconceptions. Likewise, some studies conducted on pre-service teachers alone also showed that 
they had many misconceptions on the subjects of motion and force (G6nen, 2008; Yumusak, 2008; Tung et al., 
2012). Carried out on 267 pre-service physics teachers, GOnen (2008) found out that pre-service teachers had 
grave misconceptions on the subjects of inertia, gravity, gravitational acceleration, gravitational force, mass, 
and weight. Tung et al., (2012) conducted a study on 301 pre-service teachers from seven regions of Turkey. 
They determined that approximately 90% of pre-service teachers were unsuccessful in making predictions and 
explanations in regard to the question about inertia. 

International research on students’ basic physics concepts (from primary education to university) found 
similar results (Stead & Osborne, 1980; Watts & Zylbersztajn, 1981; Watts, 1982; Sequeira & Leite, 1991; Kavanagh 
& Sneider, 2007; Lair & Cook, 2011). Kavanagh & Sneider (2007) reviewed the body of research on basic phys- 
ics concept understanding by organizing the nature of the findings, beginning with studies of the youngest 
children, followed by older students, adults, and teachers. They found a diversity of misconceptions at all age 
levels. The finding that even college physics students have significant misconceptions about free fall underscores 
the importance of effective teaching at the middle and high school levels (Sequeira & Leite, 1991; Kavanagh 
& Sneider, 2007). According to Kavanagh and Sneider (2007) to promote students’ understanding of force and 
motion, science curricula and teaching should be designed to engage students in a variety of challenges involv- 
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ing the qualitative use of the science concepts before they are expected to solve quantitative problems. They 
stated that: 


“Students do not have enough opportunities to think about how Newton's laws apply in real-world contexts. The 
mathematically rich problems in textbooks sometimes mask students’ misconceptions because they can find the right 
equation and plug in numbers to get the right answer. Unless instructors can encourage their students to transport 
science learning across the boundaries of the classroom into the real world, the entire value of science education 
will be called into question. To put it differently, science educators succeed when their students carry with them the 
insights of science into their own world of everyday lived experience. Designing problems that engage students in 
applying their growing understanding of motion to real-world contexts is challenging but essential if students are 
to become scientifically literate (Kavanagh & Sneider, 2007, p. 28). 


According to Turkish Statistical Institute [TUIK] data, the number of traffic accidents taking place on roads in 
Turkey in 2012 is 1,296,634. 268,079 people were injured, and 3,750 people were killed as a result of these accidents. 
In other words, 734 people are injured and 10 people are killed as a result of traffic accidents every day in Turkey. 
Might there be a relationship between these big traffic accident losses and whether inertia, which is Newton's 
first law of motion, is understood or not? Bingham (1997, p.19) describes inertia:” It is hard to get things moving, 
especially if they are heavy, and once they are moving it is hard to make them stop.” Inertia is also described as “the 
magnitude that measures an object's tendency to remain at rest or keep moving with constant velocity.’ (Biggs, 
Daniel, Feather, Ortleb, Rillero, Snyder, & Zike, 2005, p. 692).” Based on the examination of the drivers’ faults in the 
traffic accidents that took place on roads in 2012 and involved injuries and deaths in the light of the definitions 
provided above, it is seen that the violation of right of way at the intersection, wrong direction changing maneu- 
vers, and rear-ending are the leading causes (TUIK, 2012). Analyses of reasons for traffic accidents involving deaths 
and injuries demonstrate that 88% of accidents result from drivers’ faults. Not wearing seatbelts is another fault 
of drivers and passengers. According to statistical analyses, non-use of seatbelts increases the ratio of accidents 
resulting in deaths approximately 4.5 times. 

This study had four purposes: (a) to determine how well students understood the basic physics concepts, (b) 
to determine whether public and private school students differed in their understanding of the concepts, (c) to 
analyze the science concepts and their connection to real-life context included in science curriculum and textbooks, 
and (d) to evaluate the effect of formative assessment probes integrated with extracurricular hands-on science 
activities on understanding of these concepts. Research questions: 

1. How well do students understood the basic physics concepts? 

2. Do public and private school students different in their understanding of the basic physics concepts? 

3. What degree the science concepts and their connection to real-life context was included in science 
curricula and teaching materials? 

4. What is the effect of formative assessment probes integrated with extracurricular hands-on science 
program on students’ understanding of the basic physics concepts? 


Methodology of Research 
Participants 


Participants were recruited from three public schools and one private school. The rationale of including 
students from different schools was to compare students in private and public schools on their understanding of 
selected, basic physics concepts to determine whether knowledge and misconceptions were consistent across age 
or influenced by differences in school and related home environments. The one private school in the study has a 
better educational setting than the public schools in terms of physical environment, budget, library and class size. 
The private middle school students either have achieved high test scores, winning them scholarships, or come from 
families able to pay for private schooling. The number of students from public and private schools were. 1) Public 
school 1: 31, 2) Public school 2: 32, 3) Public school 3: 54, and 4) Private school 80. Total participants included 197, 
8th grade students who had completed studying the five basic physics concepts: mass, weight, inertia, air resis- 
tance, surface sliding friction and gravity. This study was conducted with the assistance of three science teachers 
who were master students at a college of education. Two of the science teachers assisted in the study to collect 


246 


WSs 


Journal of Baltic Science Education, Vol. 13, No. 2, 2014 


ISSN 1648-3898 EFFECTS OF FORMATIVE ASSESSMENT PROBES INTEGRATED IN EXTRA-CURRICULAR HANDS-ON 
SCIENCE: MIDDLE SCHOOL STUDENTS’ UNDERSTANDING 
(P. 243-258) 


data from their classrooms to evaluate students’ conceptions about basic physics concepts. One teacher (the third 
author) participated in the study to collect data and implement the instructional intervention. 


The Design of the Study and Intervention 


This study was conducted in the second half of the spring semester after all students had completed course- 
work on the basic physics concepts from their textbook. The rationale of conducting the study in the second half 
of the spring semester was to determine the level of students’ understanding of the concepts after they completed 
study from textbooks. To evaluate students’ understanding, eight formative assessment probes were administered 
to the all students. Evaluation of the pretest data indicated that all students had poor conceptual knowledge / 
application of the five physics concepts. (The specific activities will be explained in the paragraphs below.) Using 
these results as a formative assessment, we designed an extracurricular hands-on program by developing and 
adapting activities from science activity books. These activities integrated real-life context while implementing new 
teaching techniques in one of the public schools. These hands-on activities utilized various modeling techniques 
which included videos and simulations involving authentic materials. In order to determine students’ reasoning 
behind prediction, the instructional approach “Prediction-Observation-Explanation” was modified to “Prediction- 
Explanation-Observation-Explanation’, as suggested by Keeley (2008). The extracurricular instructional interven- 
tion activities were: 1) which water bottle hits the ground first? 2) the falling orange, 3) can you turn the corner? 
4) Video: Zero gravity, 5) which egg (raw or hard boiled) stops spinning first? 6) Video: “No seat belt, no excuse’, 7) 
how to measure mass and weight, 8) the racing copy papers. 

The first activity “Which water bottle hits the ground first?” was developed by the authors. In this activity stu- 
dents were told: “two water bottles are dropped from the same height at the same time. Which water bottle hits 
the ground first? Please make a prediction and explain your answer.’ Most of the students said that the “heavier 
bottle will hit the ground first, since it more heavy.’ After discussion in the class, teacher took two bottles of the 
same size but one full and the other half full of water. The bottles dropped from the first floor to the ground. All 
students were surprised that the bottles hit the ground at the same time. They repeated the experiment over and 
over, but they observed the same result. 

The second, “The falling orange” activity was drawn from Bingham (1997). In the falling orange activity a cup, 
post card, matchbox and an orange were used. A post card was put on top of cup, and then matchbox was placed 
on post card. Finally, the orange was put on top of all. Students were asked to predict and explain what would 
happen when the post card was quickly pulled away. Then they observed that on every trial the orange dropped 
into the cup. 

After discussion, the students concluded that heavy objects have greater inertia so once they started to move 
they do not easily stop and change their direction. In addition to falling orange activity, students participated on 
the playground in the “can you turn the corner” activity. In this activity, students were asked whether they could 
turn the corner of the building while running very fast and parallel to the building. They discussed experiencing 
that while they were running very fast, they were not able turn sharply. Because of their inertia, they were not 
able to change their direction instantly. After all these experiments to teach inertia, students were asked to explain 
why over a certain speed limit a driver cannot control the car by using the steering wheel or brakes and usually 
accidents happen. Several examples of accidents were discussed in the classroom in relation to inertia. The most 
common faults in traffic accidents were related to difficulty at instant manoeuvres to change direction and instantly 
stop, while driving over a certain speed limit. In addition to hands-on activities, students also watched the videos 
about zero gravity in space and in a spaceship (http://www.youtube.com/watch?v=bWyMtJanrts). These videos 
were taken inside of a spaceship where there is no gravity ad friction. Therefore, students observed inertia easily 
by watching the actions of the astronauts. 

To promote and integrate students’ understanding of inertia in a real-world context, the teacher provided 
an activity called “Which egg (raw or hard boiled) spins?” (Bingham, 1997). Students worked in groups with a hard 
boiled and a raw egg. They placed the eggs on the tablet to spin. Students were asked to make predictions and 
explain what would happen if each egg were stopped for an instant by placing a finger on it, then released. After 
several trials, students observed that the raw egg continued spinning. After their observations with the teacher’s 
help, students decided that a raw egg has yolk in it, so even though the shell stops spinning, the yolk inside the 
egg continues to spin. This activity integrated with inertia and car crash phenomena, where cars stop quite quickly 
and passengers in the car need to be kept seated. The sharp changes in direction and stop kill, not so much that a 
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car keeps going. The car can go in terrible directions and bodies try to go straight. Therefore, if passengers do not 
fasten their seat belt, their inertia forces them to keep moving in their original direction and they may pop out of 
the window or hit each other. This activity was followed by watching a video about the use of seat belts called 
“No seat belt, No excuse” (http://www.youtube.com/watch?v=gYpuPZYrd2M). This video is a good example that 
demonstrates what really happens in a car crush. In this video, if a person does not fasten their seat belt, s/he is 
like a billiard ball and has the potential to hit anywhere. After students watched the video, they reflected on their 
understanding about the importance of fastening seat belts. 

In the “how to measure mass and weight” activity, a balance and spring scale were brought to the classroom. 
By demonstrating how they operate, the teacher facilitated understanding that a spring gets stretched by gravity. 
An electronic balance measures resistance to gravity when a weight is on it. A balancing scale balances gravitational 
force on one side with known weights against gravitational force on the side with the unknown weight. 

Lastly, students did “the racing copy papers” activity as a whole class activity. In this activity, a student worked 
a partner, and each partner had two A4 sheet of paper. They crumbled one of the paper into a ball and kept the 
other as a sheet. The teacher asked them to predict which one would hit the ground first ,;when they dropped from 
the same height at the same time. Then they were asked to explain their own prediction. After their observations, 
students revised their explanations. With this activity, students were able to experience and develop a simple 
understanding of air resistance. 

Even though students were familiar with surface friction from daily life, they tested the motion of different 
materials over a ramp made by using their science textbooks. The teacher asked them to make predictions and and 
to explane the results, when they allowed different materials such as a pencil sharpener, eraser, cloth, sponge, etc. 
to slide freely. The students were asked to compare and contrast the distances the materials they tested moved on 
the ramp. Also they explored the effects of height (angle) on the sliding behavior of the materials. 


Data Collection 


To address the first, second and third research questions, a pretest of scientific concepts was given to three 
public and one private school. The questionnaire consisting of eight formative assessment probes (4 multiple 
choice and 4 open-ended questions) was administered (See Appendix A). The first set of four multiple choice type 
probes was adapted from Lair and Cook (2011). These probes were designed to test students’ ability to predict 
real-events such as free fall and inertia. The second set of open-ended probes was designed to measure students’ 
ability to explain inertia and friction in real-world contexts. The fifth probe adapted from Middle School Physical 
Science Diagnostic Assessment-Version 2.2, Misconceptions-Oriented Standards-Based Assessment Resource 
Center for Teachers (MOSART, 2010) measures students’ conceptions about inertia. The sixth, seventh, and eighth 
probes were developed by the authors. The sixth probe measured students’ level of understanding of inertia to 
apply to real-world contexts. The seventh probe was about free fall and air resistance. The final probe measured 
students’ ability to scientificaly explain the use of seat belts. Data validity was verified by using multiple data sources: 
participants’ predictions for the multiple choice questions, written explanations on the open-ended questions, 
verbally reported explanations to the teacher (the third author) and the third author's observations of her teach- 
ing. To check reliability of the categories, the first and second author separately coded the students’ explanations 
to the open-ended questions at the beginning and at the end of the course. The inter-rater reliability coefficients 
were computed by using SPSS software. Inter-rater agreements for the pre and post probes were 87%, and 91%, 
respectively. To address the fourth research question the science curricula (Milli Egitim Bakanligi [MEB], 2005) and 
textbooks (Karaarslan, Altuntas, Zengin, & TutuncU, 2008; Tung, Agalday, Akcam, Celtikli-Altunoglu, Bagci, Bakar, 
Basdag, Inal, lpek, Keles, GUrsoy-K6roglu, & York, 2007) were analyzed qualitatively. 


Data Analysis 


The data from multiple choice probes was analyzed using descriptive statistics, means and standard deviations. 
In order to analyze students’ explanations to the open-ended questions, a rubric was developed. The coding scale 
for each concept is presented in Appendix B. Students’ answers were coded into three categories (O=represents 
no answer, wrong, irrelevant answers; 1=represents partially correct answers without elaboration; 2 represents 
integrated scientific perspective). The frequencies of students’ coded answers to the open-ended questions were 
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computed. To answer the second research question responses, Analysis of Variances (A one-way ANOVAs) were 
computed by using pretest scores. Also, the Turkish science curricula and science textbooks were analyzed quali- 
tatively in terms of including the application of theory to real-world context. To determine the effect of formative 
assessment probes integrated with extra-curricular hands-on science activities on students’ understanding of 
the basic physics concepts, paired samples t-tests were computed using pre and posttest mean scores from the 
public school 1. 


Results of Research 
Degree to Which Students Understand the Tested Physics Concepts 
To determine how well students understood the basic physics concepts (mass, weight, inertia, air resistance, 
and gravity) mean scores were computed on a 100 point scale. Table 1 presents means scores for private and 


public schools. 


Table 1. Means and Standard Deviations of the Students’ Understandings of the Concepts. 


Schools N Mean SD 
Public school 1 ot 31.00 10.60 
Public school 2 32 29.10 12.90 
Public school 3 54 18.10 11.10 
Private School 80 33.00 17.20 


Findings indicated that all students had poor conceptual understanding of the five basic physics concepts, 
especially application of their knowledge to real-world contexts. Out of 100 possible points, students in the three 
public schools scored 31.00, 29.10, and 18.10. Students in the private school scored 33.00. 

In order to determine students’ misconceptions, frequency analyses for each question were computed. For 
the first question, out of 197 students, only 9 (6% of them) chose the correct answer “both objects hit the ground 
at the same time, and their speed is the same.’ The common misconception was that “heavier balls fall faster than 
lighter balls”. For the second question, 13 (7% of the students) chose the correct answer “neither of them. They 
have the same acceleration.’ The common misconception is that heavier objects’ acceleration is greater. The third 
question tested students’ understanding of inertia. Compared to the first two questions, students had a better 
overall understanding of inertia. However, they did not have a strong understanding of inertia, because only 42 
(1% of the students) 9 = 6%, 13 = 7%, 42 = 1% chose the correct answer for the 3'¢ question “spaceship moves ina 
straight line forever”. The fourth question was designed to test students’ understanding of the difference between 
mass and weight. This question got the best overall results with 71 (6 % of the students) choosing the correct 
answer “traveling to the Moon, my mass remains the same but my weight changes.” 

Students’ explanations to the open-ended real-world context formative assessment probe about inertia and 
friction were analyzed. The result of the analysis indicated that students’ ability to explain real-world integrated 
probes were much worse than for multiple choice type probes. Students’ explanations were coded from zero to 
two. Based on this coding schema, the percentages of correct explanations ranged from 0,5% to 11,2%. Probe 
five on had the overall lowest of explanation of all four open-ended probes with only 0,5% of students explaining 
the probe correctly. This would indicate that students have very little understanding of how to explain Newton's 
First Law or inertia in a real-world context. Probe six about also had a very low explanation rate; 5,1% of students 
explained the probe correctly. The general answer to this probe was: when a tire stops, the car stops. The correct 
explanation of this probe is that a tire can stop, but the entire car can slide or bounce and crash around. Probe 
seven concerning had the overall highest of explanation rate of all four open-ended probes with 11,2% students 
explaining the probe correctly. The general misconception or incorrect explanation was that “a paper ball was 
heavier than a paper sheet.” For probe eight about, only 4,1% of the students explained the probe correctly. In 
general students know the benefits of fastening seat belts but they were not able to relate to inertia. 
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Comparison of Schools for Student Differences in Physics Concepts Understanding 


To determine whether the private school and public schools differed significantly, an Analysis of Variance (A 
one-way ANOVA) was carried out. The dependent variable was the mean of questions answered correctly about 
the basic physics concepts. Results indicated that the students’ understanding of the concepts across the four 
schools was significantly different, F (3, 197) = 12.89, p < .001. Tukey post-hoc comparisons of the private school 
with public schools indicated that private school students’ understanding of the concepts (W=33.04; SD=17.20) was 
significantly different only from public school3 (M = 18.05; SD = 17.20). Comparisons of all four schools indicated 
that public school3 (WV=18.05; SD= 11.10) was significantly different from the other three schools (public school1, 
M= 31.04, SD= 10.64; public school2, M=29.10, SD= 12,85; private school, M= 33.04, SD= 17.20). However, there was 
a significant difference among students’ understanding of the concepts at the public school! (M=31.04; SD=10.64), 
public school2 (M=29.10; SD=12.85) and private school (M=33.04; SD=17.20). Therefore, even though there was a 
difference between schools, all schools still had low conceptual understanding of the science concepts on a 100 
point scale. 


Analysis of Related Curricula and Teaching Materials 


Identify the reason for overall poor understanding of the basic physics concepts, the content of the Turkish 
science curricula was analyzed in terms of engaging students to apply these concepts in real-world contexts. The 
Turkish science curriculum does not include any objectives on the laws of force and motion intended to integrate 
theoretical knowledge into real-life events (MEB, 2005). Students are expected to investigate relationships between 
force, mass, and the motion of objects and demonstrate the effects of balanced and unbalanced forces on an object 
in terms of gravity, inertia, and friction (MEB, 2005). 

The definitions of basic physics concepts in the textbooks and the images, examples, and activities provided 
about them, and the end-of-unit assessment questions presented about them were examined in order to analyze 
the degree to which basic physics concepts were taught through integration with daily life events in those science 
and physics textbooks. In Turkish, inertia is defined as lack of activity or automation (TDK SdzlUk, 2012). Inertia is 
referred to as balanced forces in science textbooks (Tunc et al., 2007), while it is indicated as the first law of Newton 
in physics course books (Karaarslan et al., 2008). It is generally described as an object's preserving its state of mo- 
tion. The review of the images included in the middle school science textbook demonstrated that while 18 of these 
images were related to the 2"? law of motion of Newton, there were just 2 images about the first law of motion 
(inertia) of Newton. The textbooks did not give coverage to experiments that could enable students to have some 
experiences concerning inertia. For instance, there wasn't any activity demonstrating resistance of an object to 
change in its motion and direction. When the textbook of physics was examined, in order to teach the concept of 
inertia, the examples were given as: forward and backward movement of the passengers during a sudden brak- 
ing; outbound aircraft with a constant speed at a certain height; and rain drops falling towards the ground at a 
constant (Karaarslan et al., 2008). However, textbooks do not link the causes of accidents that generally occur on 
busy highways and the concept of inertia (Karaarslan et al., 2008; TUIK, 2012; Tung et al., 2007). Instead of this, the 
2nd law of Newton and the various “algorithmic problems” about this law were extensively included in the physics 
textbooks. An algorithmic problem involves solving science problems by plugging numbers into specific equations 
to get the right answers. The difference between the concept of weight and mass are extensively included in the 
textbooks with visual examples from earth, other planets, and space. 


Effect of Real-Life Integrated Formative Assessment Probes With Extracurricular Hands-On Science Activities 
on Students’ Understanding of Physics Concepts 


To determine whether the public school1 students’ understanding of the concepts changed after intervention, 


paired samples t-tests were computed using pre and posttest mean scores. Table 2 presents the means and standard 
deviations of the students’ understanding of the concepts before and after intervention at the Public school1. 
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Table 2. Mean Percentages Correct and Standard Deviations before and after Instruction. 


Public School1 Pre-test Post-test t 
N M SD M SD 
31 31.04 10.60 57.05 22.60 5.90* 
*p<0.001 


A paired sample t-test analysis indicates that the students understanding of the basic physics concepts at 
public school was statistically different, t (31) = 5.90, p <0.001. 


Discussion 


Analysis of the students’ understanding of the basic physics concepts indicated that all students had poor 
conceptual understanding of the five basic physics concepts and lack of ability to explain these phenomena in 
real-world contexts. We found that those students retained these misconceptions even after having completed 
the appropriate coursework. For example, the first and the seventh formative assessment probes were aimed to 
extract the initial knowledge that students had about the effect of mass and volume on the speed of falling to 
the ground. The proportion of students answering both questions correctly was approximately 10%. This finding 
was consistent with research studies (Koray & Tatar, 2003; Yumusak et al., 2004; Kocakulah & Acil, 2011; Ayvaci et 
al., 2012). The common misconceptions were “heavier objects fall faster and they have greater acceleration”. The 
proportion of correct answers given by students to the fifth, sixth, and seventh questions measuring their skills 
to explain real life events by using their knowledge about inertia varied between 0.5% and 11,20%. That is a very 
clear indicator that students did not understand the concept of inertia. The probe in which the understanding 
of the difference between mass and weight was measured was the question in which the highest ratio of correct 
answers was observed (71%). This higher ratio may be attributed to the fact that the difference between the said 
two concepts is emphasized very much in curriculum acquisitions and is explicitly taught through examples in- 
cluding various images in the science textbook. 

When the performances of students from different schools were compared in terms of basic physics concepts, 
significant differences were found between schools. The most successful performance came from a private school 
(33.04 %) among the schools included in the present study. Two public schools, displayed performances of 31.04% 
and 29.10%. Thus, a success difference of approximately 4% was detected between the private and two public 
schools. The difference of 4% is below expectations since private schools are better than public schools in terms 
of physical conditions, science course hours, and the number of students in classrooms. Like students in public 
schools, private school students both form and hold misconceptions, making their ability to explain open-ended 
probes much poorer than one might pre-suppose. This result is consistent with previous studies (Yumusak et al., 
2004; Kocakulah & Agil, 2011; Lair & Cook, 2011). For instance, it was found that although classrooms had better 
conditions, there were more course hours, and courses were more test-solving focused in private schools, students 
from these schools marked incorrect choices, or made incorrect explanations, or made no explanations in three- 
fourths of the questions about explaining and applying basic physics concepts to real life. This result reveals that 
students, regardless of school types, generally have low success in applying basic physics concepts to real life 
events. This is likely because; conceptual understanding cannot be achieved more drilling/testing. Many studies 
have demonstrated that these kinds of practices do not bring about any conceptual understanding among students 
(Stead & Osborne, 1980; Watts & Zylbersztajn, 1981; Watts, 1982). 

The authors’ review of the Turkish science curriculum revealed that it gives a little coverage to provision 
concerning the application of basic physics concepts to real- world events. The provision included in the science 
curriculum determine the content of textbooks. The review of textbooks showed that there are many deficiencies 
in the integration of the above-mentioned concepts with real life (Tung et al., 2007; Karaarslan et al., 2008; TUIK, 
2012). What is common is the traditional teaching approach that attempts to make students acquire theoretical 
knowledge through solving various algorithmic problems rather than applying the laws of motion introduced by 
Newton to real life events (Tung et al., 2007; Karaarslan et al., 2008). This approach is in conflict with the recom- 
mendation that students should first learn how to solve qualitative problems concerning laws of nature, and then 
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be provided with teaching practices aimed at the solution of quantitative problems (Kavanagh & Sneider, 2007). 

A post-test was administered at the end of the extra-curricular program and the results showed a significant 
increase in the scores of the students who participated. Evidence demonstrated that students who received extra- 
curricular hands-on science instruction integrated with formative assessment probes significantly outperformed 
and that result corroborates earlier studies’ that found success was related to the method of course delivery (Hogan, 
Nastasi, & Pressley, 2000; Ruiz-Primo & Furtak, 2004; Yin et al., 2013). Similar to the findings of this, Yin et al., (2013) 
found that the middle-school students who were in the experimental group that received formative assessment 
got higher scores on general achievement tests and especially the performance assessment than the students in 
the control group. 


Conclusions 


The findings of the study strongly suggest that conceptual understanding cannot be achieved by students 
through direct transfer unless the prior knowledge and experiences, both valid and invalid, of students concerning 
basic physics concepts are put in the focus of the lesson. In science teaching practice involving direct transfer from 
the textbook, students just memorize basic physics concepts and algorithmic problem solutions, but they fail in 
explaining and applying their knowledge about the concepts to real-life events. The results suggest that current 
practices of teaching only from science textbooks is not enough for students to gain a working understanding 
of basic physics concepts. In this sense, it is necessary to generalize the use of formative assessment — a teaching 
approach where assessment and teaching are integrated into one another — over all science classes. 

That schools generally have low success in applying basic physics concepts to real life events indicates that 
different school types adopt the same traditional teaching approach in which concepts are conveyed to students 
directly and students are tested via level determining exams rather than offering a learning experience aimed at 
conceptual understanding. The conceptual understanding cannot be achieved by students through more drilling/ 
testing; the approach to education provided at schools appears far less than optimal. 

The findings that science curriculum and textbooks give little coverage to the application of basic physics 
concepts suggest that science curriculum would be much enhanced by requiring students to learn key concepts by 
applying them to real-life events. While not an issue originally intended for this study, it is an attention-requiring 
situation that although there is a direct relationship between traffic accidents and the laws of motion of Newton, it 
is not included in the science and physics textbooks in Turkey. Many traffic accidents involving injuries and deaths 
take place both in Turkey (TUIK, 2012) and across the world due to the fact that inertia, one the laws of motion of 
Newton, cannot be understood or is not taken into consideration in traffic. This practical issue illustrates that science 
concepts are valuable to every one, not only to scientists. Thus, science and physics textbooks should definitely give 
coverage to the relationship between the laws of motion and the causes of traffics accidents as well as the rules to 
be observed in the traffic. In other words, it is of vital importance that the relationship between traffic accidents 
and the laws of motion of Newton should be made visible to students within the scope of science and physics 
courses. This is only one of many possible practical appllications of physics concepts that textbook authors and 
teachers might use for exciting students to learn and to improve their lives as a result. Such applications answer 
the ever-present question: Why is this boring stuff important to me? 

The findings that science curriculum and textbooks gives a little coverage to the application of basic physics 
concepts suggest that science curriculum would be much enhanced by requiring students to learn key concepts 
by applying them to real-life events. It is an attention-requiring situation that although there is a direct relationship 
between traffic accidents and the laws of motion of Newton, it is not included in the science and physics textbooks 
in Turkey. Many traffic accidents involving injuries and deaths take place both in Turkey (TUIK, 2012) and across the 
world due to the fact that inertia, one the laws of motion of Newton, cannot be understood or is not taken into 
consideration in traffic. This practical issue illustrating that science concepts are valuable to every one, not only to 
scientists. Thus, science and physics textbooks should definitely give coverage to the relationship between the 
laws of motion and the causes of traffics accidents as well as the rules to be observed in the traffic. In other words, 
it is of vital importance that the relationship between traffic accidents and the laws of motion of Newton should 
be made visible to students within the scope of science and physics courses. 

A significant increase in the scores of the students who participated in the extra-curricular hands-on science 
program suggests that academic activities just based on science textbooks are not enough for students to under- 
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stand the basic physics concepts, and there is need to include a variety of hands-on science activities These hands 
on activities utilized various modeling techniques which included videos and simulations involving authentic 
materials. To increase understanding, more time may have been needed for investigation and reflection. In order 
to improve Turkish students general scores, changes of approaches in education, more constructivist practices 
in newly developed curriculum, and the use and diffusion of technology are definitely needed. In addition, Turk- 
ish teachers may conduct their own action research that might focus on using formative assessment and related 
probes to improve their students’ conceptual understanding of basic physics concepts. that is not very common 
among Turkish educators that might also have been needed. Thus, designing formative assessment probes that 
engage students in applying their understanding of basic physics concepts to real-world contexts is essential to 
educate them to be scientifically literate. 
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Appendix A: Formative Assessment Probes 


m 2m 
Note: First and second questions will be answered the information below: x © Y © 
1. After studying a dynamic unit in school, a group h h 


of students got into a discussion about the follow- 
ing question related to free fall. What will happen 
when the two small objects are dropped at the \ ~ 
same time at the same height? Object Y is twice as 

massive as the object X. This is what they say: 





Ali: Both of them hit the ground at the same time, but object X will be moving faster than the object Y 
when they hit 
Mehmet: Both balls will hit the ground at the same time, but the object Y will be moving faster than object 


X when they hit. 
Ayse: Both objects hit the ground at the same time, and their speed is the same. 
Omer: Object X hits the ground first, since it is heavier 


Circle the name of the student you most agree with. Explain why you think that is the best answer. 


2. Students also argue which ball has the greater acceleration during this fall. Here is what they say: 


Ali: Object X 

Mehmet: Object Y 

Ayse: Neither of them. They have the same acceleration 
Omer: Both of them drop with constant speed to the ground 


Circle the name of the student you most agree with. Explain why you think that is the best answer. 
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3. InEarthand Space unit, a student wonders 
what would happen if a spacecraft left the 
Solar System and is in empty space where 
there are no forces. 


Circle the answer that best matches your 
thinking. 





A. Move in a straight line forever 

B. Coast to stop 

C. Move in a circular orbit around the galaxy 

D. It is impossible to tell what the spacecraft will do 

Explain your thinking. Describe the reasoning you used for your answer 


4. You wonder what would happen to your mass and 
weight while you are traveling to the Moon. Circle the 
answer that best matches your thinking. 


A. My mass remains constant but my weight changes 

B. Both my mass and weight remain the same 

C. My mass changes but my weight remains the same 

D. Both my mass and weight decrease 

Explain your thinking. Describe the reasoning you used for your answer 





5. When studying force and motion, a group of 
students points out that a car will stop rolling 
on a flat surface unless you keep the engine 
onandin gear so that it continues to provide a 
force to run the wheels. They cite this situation 
as a violation of Newton's First Law of Motion 
(an object in motion continues in motion | | | 
unless acted on by an outside force). SS EEE ESS A 
Do you agree or disagree with this group of 
students? Please explain your reasoning. 
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6. Overacertain speed limit, by pressing the brake 
pedal, the tires can be stopped but the car will not. 
By turning the steering wheel, the tires direction 
can be changed, but the car will not. 
Do you agree or disagree with these statements? 
Explain the reason for your answer. 





7. Whatwill happen if two identical papers (one is a 
sheet and the other is crumbled) were dropped 
from the same height at the same time? 





Explain the reason for your answer. 


The vectors in the figure is not adequate. They 
should be different Feb GP Ue Shas Bele Ob oF 


8. Fastening seat belts is a mandatory rule that we 
have to obey when we ride in a car. Beyond being 
a rule, please explain scientifically the reason for 
fastening the seat belt in the car. 


FOR YOUR SAFETY 
AND COMFORT 
FASTEN SEAT SEL! 
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Appendix B: Scoring Rubric for Open-Ended Probes 


2: Integrated with scientific perspective (True prediction and explanation) 
1: Partially correct or has no elaboration. 
0: No response, wrong, or irrelevant answer. 


Score Question 5 


lf s/ne can explain the reason that the car stops with two or more friction forces such as air resistance, tires and surface, or any 


e moving parts in the car. 
1 Is s/ne can explain the reason that the car stops with only one friction force such as tires and surface or air resistance 
0 No response, wrong, or irrelevant answer 
Score Question 6 
2 Over a certain speed limit, the car’s inertia will be bigger so by pushing on brake pedal, the tires will stop turning but the car will not 
stop instantly and it will slide. Also, by turning the steering wheel the tires will turn but the direction of the car will not change. For 
instance, in a sharp curve, if a car’s speed is very high, it usually skids off the road. 
1 lf s/ne only gives an example about the consequence of using brakes or steering wheel over certain speed limit such as sliding, 
skidding or turning over 
0 No response, wrong, or irrelevant answer. 
Score Question 7 
2 lf s/ne can true predict that a paper ball will hit the ground first. Also s/he can explain that the air resistance acting on a paper ball is 
smaller because of it has less surface area than a sheet of paper. The net force pulling the papers down is equal to the gravitational 
force minus air resistance force. 
1 Is s/ne can make a true prediction that “the paper ball hits the ground first” but they cannot relate their explanation with air resistance 
as an opposing force to the motion of the paper. 
0 No response, wrong, or irrelevant answer. 
Score Question 8 
2 lf a person is driving a car at a certain speed, s/he has the same speed with the car. In a car crash, the person without a seat belt on 
will keep moving the same speed and the same direction. This will cause injuries or death in the car accident. However, if the seat 
belt is on, the seat belt will stop his motion and keep him seated in the car. 
1 lf s/he can make a true prediction about the consequence of not fastening seat belt in a car but cannot relate it to inertia. 
0 No response, wrong, or irrelevant answer. 
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